When writing scientific literature and articles using a computer, users often must input various and sometimes complex mathematical expressions. Today, a user has to input the mathematical expressions in an indirect manner. For example, FIGS. 1A and 1B show two systems for inputting a mathematical expression: a structured expression editor, such as Equation Editor by Microsoft® Corporation of Redmond, Wash. (FIG. 1A); and an expression descript language, such as LATEX Equation Editor (FIG. 1B). FIG. 1A shows a large tool box 101 which contains items 105-1-105-N corresponding to various mathematical symbols and structures. The input of expressions may be laborious for some as a user has to find the proper symbol or structure from the groupings. FIG. 1B shows a second system that is oriented more towards an expert in the area of mathematical expression script languages. Users have to become experts of the script language before they may utilize the language to input expressions freely. Both systems are designed for the mathematical expression to be inputted by a keyboard.
The use of an electronic pen and/or stylus input device is a more natural method for users to input mathematical expressions. The tablet style computer allows a user to enter handwritten notes; however, mathematical expressions have not been recognized with high accuracy by existing handwriting recognition software packages. A need exists for handwritten mathematical expression recognition to enable pen-based input. Comparing to printed expressions, more ambiguities exist in handwritten expressions. Firstly, it is hard to differentiate symbols from each other just by using shape information. For example, ‘X’ is very similar to ‘×’, such as for designating a multiplication operation. Another typical example is a ‘dot’. When a dot is located at a position of a subscript, it is a decimal dot. However, when the dot is at a mid-level position, it is a dot operator. Secondly, there are many uncertainties in a layout structure. For example, a numerator may expand to a region outside of a fraction line because there is not enough room above the line.
With the rise in use of the tablet style computer, applications are being created and/or updated to implement handwritten annotation recognition. However, handwritten text recognition and ink document analysis are the only recognition and analysis systems enabled in a freehand input system. Handwritten mathematical expression recognition has not been available yet.